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a design and technology workshop. Study A was a multiple method qualitative 
case study that included semi -structured interviews with 10 exhibit 
development staff members and 25 visitors, exhibit content analysis, text 
analysis of more than 3,000 visitor comments, and a scan of boxes of exhibit 
development documentation. Study B used a naturalistic inquiry approach to 
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sort of meta analysis, provided insights into the nature of informal science 
learning in the museum. The dialogical reflection of the researchers resulted 
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inquiry competencies into available professional museum competency schemas. 
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INTRODUCTION 



Adults and children are exposed to many potential educative sources in their daily 
activities, and it is from such out of school sources that most people learn for most of 
their lives. 

Lucas, 1983, p. 1 

This paper will briefly discuss two independent museum education case studies, 
conducted by educator/ researchers at the same large science centre in Toronto. We will 
discuss the relative methodologies used for the two studies, and then focus on the 
collaboration we established for making sense of the findings in terms of broader 
implications for the educational role of the museum and the professional development of 
its practitioners. It is our hope that informal learning researchers and educators may be 
encouraged to reevaluate studies conducted at their own informal learning setting to 
determine whether or not those studies might in fact be a useful source of comparative 
data for further research or reflection. 

Our paper is intended for anyone who has an interest in informal learning/ learning in 
non-school settings, or museum education research methods. Our findings will also be of 
interest to museum staff involved in education, exhibit development or public 
programming. More generically, this paper will interest anyone wishing to expand his/ 
her understanding of reflective practice. 

The Case Studies 

Two very different studies in educational design were recently conducted 
independently at a large science center in the Toronto area. One study (“study A”) looks 
at exhibit development practice and its relation to casual visitor understandings of key 



concepts in a particular exhibition, and is being conducted for Livingstone’s doctoral 
dissertation. The study Lemelin was associated with (“study B”) engaged science centre 
educators in critical reflection on their pedagogical practices pertaining to the delivery of 
a design and technology workshop. 

Study A (in progress) is using a multiple method qualitative case study design to look 
at the development and communicative effectiveness of an issues-based science centre 
exhibit. Data sources for this study include: semi-structured interviews with exhibit 
development staff (N= 10) and visitors (N= 25), exhibit content analysis using two 
published protocols, and text analysis of more than 3000 visitor comment sheets plus 
material gleaned from a scan of 7 banker boxes of exhibit development documentation. 

It has been found that the educational goals of the exhibit in question were clearly 
defined from the outset of the development process. During interviews, for example, in 
order to explicate the intended message of the exhibit as they understood it, exhibit 
development staff were asked to call on memory to “describe one the exhibit displays 
with which you were involved in development.” In response, most staff interviewed 
focused their discussion on the content of a specific display. Somewhat less attention was 
paid in these recollections to how a display functioned, or how it fit into a dominant 
exhibit narrative. Thus, visitor learning does not seem to have been as clearly 
conceptualized during the development process. 

Visitor comments (both written and in interviews) demonstrate mixed results as to the 
efficacy of the exhibit displays in communicating key concepts in the absence of 
additional assistance from a museum educator/ animateur. Nonetheless, visitor responses 
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do demonstrate considerable engagement and contemplation of abstract exhibit concepts 
on the part of many visitors. 

Study B used a naturalistic inquiry approach (Guba and Lincoln, 1988) and emergent 
research design. Personal construct theory was looked to in the development of a case 
study that describes the science centre educators’ reflective pedagogical practice. 

Through a series of workshop observations and semi-structured repertory grid interviews 
it was found that the educators' primary role is one of support to the public school system. 
In that capacity, educators developed programmes that are closely linked to provincial 
science curricula. It was found that the science centre educators are often perceived by 
visiting schoolteachers as "expert scientists" or “expert technologists”. Museum 
educators also self-define their role as including research into what technologies are 
available, learning about scientific investigation, and making new technologies available 
to school visitors (Bencze & Lemelin, 2000). 

Recognizing a need for an emphasis on doing science and technology, educators at 
the science centre introduced a design and technology workshop for elementary clientele. 
These sessions were planned to engage students in open-ended, student-directed design 
projects, after having taken part in apprenticeship lessons and activities. Bencze 
&Lemelin (2000) reported that this innovative workshop succeeded in allowinf students 
to take control of their learning. The educators were successful in shifting the locus of 
control to students and implementing the workshop as planned. In spite of these 
successes, however, two factors acted as barriers to this shift in the knowledge 
construction in education programmes at this museum. 
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As Bencze & Lemelin (2000) report, students’ designs were often limited by the 
technologies available to them and the instruction offered within the first half of the 
workshop pre-determined student designs ( technological determinism). Also, while this 
science centre’s chief pedagogical mandate is to support formal science and technology 
education by tailoring its educational experiences to match needs of and practices in 
schools, the workshop suffered somewhat from a general disconnectedness between 
formal and informal education. Finally, the museum did provide schools with a previsit 
resource package on the nature of design and technology, yet most students arrived 
unprepared for the workshop. Consequently, the museum educators expressed concern 
about the degree to which the workshop was being integrated into regular school 
programming, and considered ways to improve the situation. 

The Collaboration 

In the Fall of 1999 the authors established a peer mentoring partnership within our 
doctoral program in Curriculum Studies. Having both worked as museum educators, we 
drew on our experiences to problematize and illustrate perspectives in our conversations 
about curriculum theory. In time, our conversations also began to draw on our respective 
graduate research projects. At the same time, we shared and discussed relevant literature 
from museology, cultural studies, critical pedagogy and curriculum. 

Having established a collaborative ethic through this process, we began to see the 
possibility for a similar approach to research and professional development in the 
museum. Having previously witnessed the problem of research reports collecting dust on 
museum shelves soon after publication, we wondered in particular how education or 
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audience studies might best speak to one another and so expand their relevance to 
museum professionals. Although a number of OISE research projects were being 
conducted contemporaneously at the same science centre, these projects were not 
formally sharing information. Interested in exploring the potential for some sort of meta- 
analysis in broadening the relevance of our research findings, we decided to report on our 
collaborative experiment. 

CONTEXT AND THE STATE OF THE FIELD 

Policy documents such as Excellence and Equity (AAM, 1992) write public education 
as an institution- wide objective of museum practice. Scholarship (principle 4), 
collaboration (principle 6) and professional development (principle 9) are among the 10 
principles Excellence and Equity suggests are central to an action plan to answer the call 
to greater accountability in public service. Although objectives for implementing such an 
action plan are likewise offered, the principles and recommendations demonstrate an 
introspective vision for change. One objective offered within the scholarship principle, 
for example, is to (better) “explain the important role of research in museums to the 
public through exhibitions, programs, publications, and electronic media” (AAM, 1992, 
p.18). It is clear from the explanation of the principle and the other objectives offered that 
it is collections research that is implied here. Research on visitor learning and inquiries 
into what the public actually wants in a museum experience are not part of the vision 
offered. Nonetheless, the need for reflective museum practice (Worts, 1990; Teather, 
1991) and research (reflective or not) on/ in museum education has been recognized 
(Woollard, 1999; Roberts, 1997). 
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In our experience of museum educational discourse, we have witnessed the problem 
of compartmentalization of knowledge according to particular disciplines (e.g.. 

Education, Anthropology, Botany) or museum-types (e.g., science centre, art gallery, 
zoo). The average visitor's approach to museum knowledge, however, is interdisciplinary. 
The demand for greater public accountability in museums necessitates the questioning of 
traditional interpretive strategies in order to ensure the museum is engaging the visitor 
more on their own terms. In order to address the needs of diverse visitor “interpretive 
communities” (Hooper-Greenhill, 1999, pp. 13-14), museum educators need to form their 
own practical interpretive community. “A greater knowledge of the interpretive strategies 
employed by different communities of visitors would be helpful. This, of course, means 
in-depth research work” (Fish cited in Hooper-Greenhill, 1999, p.14). So, while there is 
new recognition of (1) the educational role of the museum, and (2) the communicative 
role of the museum educator, there still exists a gap between this theoretical 
conceptualization of museum education and actual front-line educator practice. 

The following examples illustrate the fact that this transformation is not yet complete. 
Our own “in-service” training as front-line museum educators did not encourage dialogue 
or reflection on a vision for the educational role of a museum as a whole. Rather, we both 
received site-specific training that largely entailed modeling our practice on that of more 
senior educators. In this process, our prior teaching experience and training got us ‘in the 
door’ but was not treated as relevant to the job. Further, we found few opportunities for 
professional development, sharing notes with our peers, or exchanges with staff from 
other departments. Thus effectively isolated from a community of practice, the situation 
was exacerbated by limited exposure to professional/ research literature. Within the same 
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context, however, education department managers do have access to professional 
development and interdepartmental contact. 

The hope for widespread implementation of Excellence and Equity, or similar policy, 
assumes certain standards in museum practice are ubiquitous. National and international 
museum associations have recently established standards and a common language for 
professional development in the form of professional competency frameworks. Examples 
include: the Canadian Museum Association’s (CMA) The Workforce of the Future (1997), 
and the International Council of Museums’ (ICOM) Curriculum Guidelines for Museum 
Professional Development (2000). These lists of competencies are for use across the 
various museum professions and over the course of a career. Both have “shared” (CMA) 
or “general” (ICOM) competencies— ones all museum staff are expected to acquire— 
including research, and evaluation. ICOM also defines general “museology” competencies 
for all staff. 

Under the heading “evaluation methods”, ICOM lists the following general 
competencies: analysis of data, data collection, project design, purpose, and report 
methods. Under “research” is listed: ability to seek out and acquire new information, apply 
learning to tasks, critical thinking, methodology. The CMA’s shared competencies include 
similar skills and are further distinguished by five levels of proficiency— from “most 
general level to the expert level”, where the more expert the level, the closer the 
competency comes to a specialist functional competency (1997, p.l 1). 

Within the ICOM public programming specialist competencies, research is not listed 
as a specialized skill, despite the call for museum education or audience research cited 
above. It is, however, within the domain of the specialist management competencies to 
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not only conduct audience research, but also engage in reflective practice. Like the CMA 
functional specialist competencies, ICOM’s specialist competencies are functionally 
more advanced than the generic ones and so, in practice, do represent a hierarchy of 
professional knowledge. In particular, this translates into managers alone having the 
reflective practice skills needed to facilitate change— an assertion we challenge. Further, 
this distinction contradicts the logic of the ICOM competency model as research and 
reflective practice skills are thereby not actually shared across the museum professions. 

The skills gap implied in the professional competencies listed is echoed in the 
perceptions of the field related to Eisner and Dobbs’ in interviews they conducted with 
directors and education curators in 20 large and medium-sized US art museums. Findings 
in their 1986 report, The Uncertain Profession , demonstrate the low status of education 
departments within the museums in question, and assert that morale, training, research 
methods and salaries for museum educators all need improvement. This confirms our 
experience where front-line educators are workers who implement plans but do not 
contribute to the development of the plans. 

Implementing policy and transforming practice requires shared vision and ‘buying into 
the plan’. Where is the voice of the front-line educator in the re- visioning of the 
educational role of the museum? Are education curator/ managers solely responsible to 
operationalize the new programming principles embraced in museum policy, while also 
keeping the disparate interests of visitors as the organizing principle? Obviously, there is 
an outstanding need to bridge the gap between idealized professional competency 
frameworks and the need for advice as to how to operationalize the necessary changes in 
interpretive strategies. 
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OBJECTIVES FOR THE COLLABORATIVE ANALYSIS 



By discussing our respective studies undertaken at one science centre, we endeavored 
to draw a more complete picture of the current educational role/s of one public museum, 
by developing a sort of meta-analysis (c.f. Scott, 1998) through co-reflection. We saw the 
case studies as complimentary as one looked at school programming and the other at 
exhibit programming. Further, we felt collaboration between different departments in the 
institution would be a step toward better understanding the nature of informal science 
learning in this science centre. We believe the potential of such partnerships goes beyond 
the project’s ‘nuts and bolts’ to include how people from different educational cultures 
might come together to develop innovative ways of presenting information to visitors of 
all ages and backgrounds. 

In sharing notes on our relative research projects, our collaboration had a number of 
objectives. These were: 

1. To compare the educational roles of museum educators with those of other museum 
staff (i.e., exhibit development staff) who help to shape the visitor's museum 
experience; 

2. To offer suggestions regarding ways to operationalize change in museum education, 
by adopting research and reflective practice as GENERAL COMPETENCIES rather 
than specialized functional ones. This is key in opening up the responsibilty for public 
education across the museum; 

3. To experiment with new methods of analysis in museological/ museum educational 
research. 

The opportunity to experiment with new methods of analysis precipitated this 
particular inquiry. It is the exploration of possible ways to better implement the 



professional museum competency guidelines for museum education practice that we 
focus on below. 

THEORETICAL FRAMEWORK 

Despite the number of studies that in some way look at museum learning, the 
definitions of learning used tend to be "unique, idiosyncratic and often implicit" 
(Leinhardt & Crowley, 1998). Many studies demonstrate the importance of the idealized 
visitor experience in the conceptualization of exhibits and programming (Falk & 
Dierking, 1992; Hein, 1998). Further, the institutional intentions, definitions and theories 
of what constitute museum education or museum learning are usually vague if defined at 
all. There is, after all, no universal definition of the educational role of the museum; there 
are still many museums and galleries where education is regarded as no more than the 
“provision of formal - often very academic - lectures or guided tours” (Boy lan, 1999). 

In our work, social constructivism (e.g.. Lave and Wenger, 1991) offers a model for 
how museum learning operates. Study A was also informed by mass communications 
theory (Hall, 1993; Hooper-Greenhill, 1995). For the content analysis conducted in study 
A, feminist poststructuralist (e.g., Harding, 1998) and cultural studies (e.g., MacDonald, 
1996) literatures were looked to in the interpretation of issues attached to the gendered 
and scientific representations displayed. Study B used personal construct theory (Kelly, 
1963) to help describe museum educators’ reflective pedagogical practice. 

In our collaboration we considered both studies in terms of what the findings could 
offer in relation to developing museum professional literacy. Linking the principles of 
reflective practice (Schon, 1983), participatory research (e.g., Patton, 1997, pp. 87 ff.) 
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and action research (Kemmis and MacTaggart, 2000) enabled us to conduct critical 
reflection in a community of learners. Here we see the espousal of general research and 
reflective practice competencies and their adoption by educators as empowerment, 
comparable to Friere’s description of conscientization in developing adult basic literacy 
(1970). 



METHODS 

Collaborative qualitative inquiry is recognized within social research as both a 
methodological decision and a form of community development (e.g., Reinharz, 1992; 
Patton, 1997). Voices supportive of collaborative innovations in museological research 
include Krapfel's interest in hybrid inquiries (1998, p.12), and Bailey's (1998) recognition 
of "cross-fertilization" during museum educational inquiry. In a similar spirit of 
collaboration, we conducted an experiment in reflective analysis and interpretation, 
whereby findings for two independent case studies at the same science centre were 
considered for broader implications for educational practice in museums. Through this 
process we developed dialogical reflection. 

Over the course of our collaboration, methods used included: 

1. Sharing notes on our relative studies, 

2. Comparing literature sources on informal science learning/ museum education 
research, 

3. Undertaking a joint lit search on museum professional development and 
participatory inquiry, 

4. Tape-recording meetings, and 

5. Through a dialogical process shared insights into the professional development 
issues raised within the two case studies discussed. 

O 
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FINDINGS 



The following excerpt is taken from a conversation we had over the course of one of 
our tape-recorded meetings. We discussed definitions of what reflective practice entailed 
for us as museum educators and educational researchers. 

P: “Maybe we should start with a definition of what we mean by self-reflective because there’s been a lot 
of talk about that. Doug Worts was writing about reflective-practice and Teather too, but they’re just of 
gesturing nice ideas. Not situated in specific examples and saying ‘you should be doing this and this and in 
this situation there are differences.’ It’s sort of a universal statement saying ‘think about this.’ Not even 
going to the level that Roberts does with a specific example. So that could be our starting point, going 
through a history of the dialogue about reflective practice. There’s been a lot of attention to Schon, but that 
hasn’t necessarily been built into Museum Studies curricula for training educators.” 

N: “But she (Roberts) acknowledges the fact that even within the educators, in the same institutions, there 
is a variety of training, philosophies, and goals for education, so with such disparities, what do you do? 

And she is sort of stating the fact, she doesn’t offer any suggestions. I mean what we could work on here is 
maybe a working model, is that too ambitious?” 

P: “Well, we could, well maybe not a model, because we want tobe case specific and question specific, and 
be sensitive to a variety of choices in terms of a variety of techniques and that sort of stuff. And 1 don’t see 
that in a lot of museum research. There’s a cookie cutter approach where people are satisfied to have a 
consultant come in and cut and paste from their old reports, do whatever it is they want them to do for the 
cheapest amount, but they don’t necessarily use or understand and take up the results of that research. It’s 
almost just to have the report on the shelf to say you’ve done some research. It’s not a living exercise.” 

“Sometimes people out there will just do research because it’s required by a funding body. You have to do 
some evaluations to see if what you did was okay... or successful. Or the way that research is done is not 
questioned, the results are all that are looked at. It’s not brought into the type of programming that is done, 
not as far as I can tell anyways... I mean say there was something done in the education department at the 
ROM, if a good number of the teachers had been involved in the study, enough of them would be around so 
that they’d remember if not the details, then at least that there is a report sitting on the shelf. So there would 
be more institutional memory and it would be more of a living phenomenon and you would have some 
continuity. In part because they would be part of the process and in part because the questions would have 
been more meaningful.” 

N: “And if you had a participatory model, I mean based on successful studies that have been published, it 
creates a habit of mind. You’re constantly questioning your practice and reflecting on what you’re doing ... 
and making changes.” 

P: “And part of that process has to do with what is it you all mean when you’re sitting around that table? 
There’s going to be a learning curve as far as when you say research what do you mean. I mean, you can 
compare the competencies almost with this sort of a process. Those competencies don’t really speak to me 
as living phenomena. These are the result of a committee sitting down and saying here are some nice ideas. 
But I wonder how many of them do research on a regular basis given those sorts of statement. That level 5 
is not a level 5 (CMA). It’s not the ultimate goal... unless the level is what everyone at every level should 
aim for in every day practice.” 




Through such discussions, we challenged one another’s assumptions and analyses, in 
pursuit of a richer understanding of educational practice/ the educational role in 
museums. Both researchers benefited from this collective reflection in a number of ways: 

1. We each gained analytical skills and insights into the research process; 

2. We each developed a broader understanding of educational practice within the case 
study site. This broader understanding would have been impossible to attain 
individually given the same timeframe and resources; 

3. We bridged the theory/ practice divide in our conversations, allowing us to both make 
sense of our past experience in a theoretical way, and inversely, to critique museum 
policy/ museology curricula based on our grounded knowledge of the field. 

The academic partnership we had previously established was enriched through the 
collaborative experience. Establishing a new co-learner partnership for such an exercise, 
however, would likewise have been a benefit to both participants. As graduate students in 
a marginal field of educational inquiry we both found community. Just as action research 
is negotiated in a community of collaborators, this form of reflective practice happens in 
community. 

The most significant insight we gained in terms of practice related to the mapping of 
inquiry competencies into available professional museum competency schemas. This 
exercise highlighted the gaps in the ICOM and CMA frameworks, whereby analytical 
skills in front-line staff are not recognized or valued-- echoing the educator 
disenfranchisement attested to by Eisner & Dobbs (1986) and ourselves (see above). This 
may explain the double standard whereby museums broadly self-define as educational 
institutions, but may not see museum educators as having broadly defined educational 
responsibilities. We see potential for this type of inquiry to contribute to museum 
professional literacy while empowering museum educators — along the lines of Friere’s 
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conscientization — while also responding to the call for professional development and 
reflective practice, in a manner other museum educators might emulate. All museum 
personnel involved with the public in some capacity "...need to collaborate with 
colleagues in similar circumstances on practical research.... which can be published or 
used for the basis of discussion with a larger professional group” (Woollard, 1999, 
P-145). 

As graduate students, we also gained practical skills in collaborative research. Our 
understanding of educational practice was greatly enriched through the comparative 
interpretation, that was in effect, also a further form of triangulation. In comparing 
qualitative case studies, provided data standards and research questions are adequately 
accounted for, a descriptive richness almost comparable with a full-blown ethnographic 
study can be made possible. 



LIMITATIONS 

Our analysis was limited by the fact that it was not planned from the outset of the 
respective case studies. The case studies are, therefore, not as complimentary as they 
might otherwise have been. This situation does, however, mimic a realistic museum 
situation, where individual studies or consultant’s reports are stand alone and may not be 
shared very widely or long. 

In the effort to situate our conversations and methods within museology, we were 
limited by the fact that the existent literature is still largely advocacy-oriented (pragmatic) 
in nature, and not theoretical. In order to engage with pre-existing theory, we have had to 
draw on methodology and education models not designed with the museum field in mind. 
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Worse, the discipline-based research traditions well-established in the museum do not 
tend to involve institutional analyses or reflexivity in museum practice. Thus, our analysis 
and suggestions might be resisted or even ignored, as it would be seen by a number of 
museum professionals as irrelevant to their work. This sort of exercise, however, might 
be taken up by other museums/ museum researchers in response to the call for 
professional development that allows museum practitioners to develops skills and 
knowledge to contribute to professional understanding in the museum as a whole 
(Woollard, 1999). 



SUGGESTIONS 

While it may be unusual for two independent case studies to occur simultaneously in 
one museum, past studies/ reports may likewise provide useful starting points for 
inquiries into museum practice. As 'snapshots' past reports can offer some institutional 
memory to new management employees, remind one that what seems static does in fact 
change, or raise questions about how things might be done differently in the future. 

The comparison and contrast of assumptions and findings we experienced suggests 
similar insights are possible in any sharing notes on processes and results of museum 
research. To reflect on one's own work alone is not nearly as productive as having two (or 
more) opinions , plus the opportunity to compare and contrast results. Where the results of 
a study or change in practice are to have institution-wide implications (e.g., related to 
museum mission), consider other 'nontraditional' institutional stakeholders for your 
research and include them in the dialogue early on for a more holistic treatment of issues. 

Finally, the implicitly hierarchical approach to museum competencies demonstrated 
by curricula such as the ICOM and CMA frameworks should be reconsidered. Enacting 
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the plan to be responsive to the public in new ways draws on the skills of museum 
workers across the institution, at all levels. Reflexive inquiry, therefore, should be 
enshrined as a valued skill across the museum professions. The recognition of research 
and reflexive inquiry as ‘pan-museum’ professional skills, we suggest, would be a useful 
first step in operationalizing the various recently established sets of museum 
competencies. 

A corollary of this need to embrace inquiry skills across the professions is the need 
for educators to assert their expertise outside of the education department per se. Blais 
(1995), for example, found educators acknowledging change but not yet convinced about 
using their expertise outside their traditional field of work. This suggests the need for 
education managers to encourage educators to bring their expertise and experience into 
the dialogue on the changing public role of the museum. Where educators lack reflective 
practice skills, they can be empowered through staff training. 
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